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EXHIBIT A 



Clinical Applications off- 
Algorithms 



Robert Patterson, M.D. 

This paper is based on presentations made at the 1 995 American Psychiatric Association 
annual meeting in a symposium "Algorithms for the Pharmacotherapy of Schizophrenia" 
and at the Veterans Administration Hospital, Bedford, MA October 1995. 

I've become involved in the process of computerizing algorithms about the pharmacologic treatment of 
psychiatric patients. This experience focused my attention on the challenge to create computerized 
algorithms that will have maximum positive impact on the care of patients. I will discuss several aspects 
of this challenge: What is it that makes someone an expert? What are the common cognitive errors of 
people making decisions? What are the advantages of algorithms and especially computerized . 
algorithms? What needs to be done to make computerized algorithms widely used? And, What are the 
prospects for refining algorithms through user feedback - probably on the Internet? 

Studies of Decision Making as a Cognitive Process 

There is a large and very interesting research literature about 
how people make decisions "under uncertainty" as its called - 
that is decision making under typical clinical conditions where 
information about the patient is incomplete, and scientific 
knowledge is also limited and usually does not apply exactly to 
the specific case at hand (Kahneman et al 1982, Dowie and 
Elstein, 1988, Williams, 1982). 



swat,: 



What Makes Someone an Expert? How does an expert solve a 
problem differently from a novice? It appears that experts do not 
use more pieces of information about a case than novices but 
they have much more knowledge about which factors are 
important in a particular situation. In addition to knowing which 
facts are important, they have greater knowledge about how 

much weight to give to each relevant fact. In a few moments we 

will, however, examine how even experts are not as good at combining those weighted factors to reach a 
decision as they are at recognizing which factors are important. 

Seeing large numbers of very similar cases gives experts the opportunity to improve their skills by 
learning the probabilities of various outcomes and their dependency on particular situations. We will 
address later how ordinary clinicians may together achieve this advantage now only available to experts. 
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"He claims to be a specialist, but I think he just has a one-track mind. " 

Thinking of a typical clinical situation can affirm these research insights: Suppose you decide to ask a 
gastroenterologist about the symptoms of one of your patients. In anticipation of talking to the 
gastroenterologist, you might ask the patient quite a few questions about his symptoms - because you 
don't know which answers are going to be important When you go to the gastroenterologist, he or she 
asks questions to learn about the* patient's general situation, then often asks a small number of questions 
about the specific problem before giving you an opinion. The expert's skill lies in knowing how to asses 
the context of the problem and then the weight to place on certain facts. 

Experts know how to break down complex problems into sequences of smaller problems. Each small 
problem requires only a few facts to resolve it. Experts approach complex problems this way because 
human information processing is relatively slow, our short-term memory has limited capacity and we ar< 
subject to frequent interruptions which would require time consuming rebuilding of mental images if 
problems were large (Moskowitz, 1988, Payne et al, 1992). 

Incremental solutions thus avoid some of our major cognitive shortcomings. However, incremental 
problem solving is vulnerable to certain kinds of errors: The order of solving sub-problems may affect 
the outcome. Another problem is that each sub-problem is resolved with a single answer but there may 
be substantial uncertainty about the correctness of the answer. The answer will be used as the input for 
the next sub-problem but we are not able to carry forward a good representation of the degree of 
uncertainty and especially we have trouble manipulating compound probabilities. After 3, 4 or 5 such 
sub-problems we may be quite lost about the overall probability of the final outcome. 




Figure 1 Decision tree for determining best treatment method for male patient with preleukemia and 
granulocytopenia. From Moskowitz et al. 

Figure 1 shows a decision analysis for a patient who has preleukemia and granulocytopenia. A decision 
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tree like this analyses the -ste^rand assigns objective probabilities to <SR alternative. The goal is to 
determine the treatment plan' which gives the patient the greatest likelihood of survival. Moskowitz et al 
analyzed the thinking of three pulmonologists who were asked to recommend treatment in this situation. 
They all recommended the biopsy path and they dismissed the empiric therapy path as unacceptable. The 
decision analysis reveals that the empiric therapy would have given the patient as good a chance of 
survival as the experts' choice. Empiric therapy would have subjected the patient to less procedures and 
surely have cost less. This figure emphasizes that experts find it difficult to assess the compound 
probability of alternative paths. 

Common Cognitive Errors in Decision Making Here we will add brief descriptions of several other 
cognitive difficulties found when people make decisions (Dawes, 1988). Experts may be somewhat less 
prone to these errors but at best they are only slightly less likely to make them. My intent is to present an 
objective overview of the difficulties we face - not make us look inept. However, unwarranted 
confidence about decisions made is a persistent finding in this research. 

Most common errors in judgement: 

9 Jgnpring base r^te - Zebras; Good response to antidepressant 

j Depending too much on ease of recall of instances - air vs auto accidents 

• Anchoring errors - Estimate commercial airplane deaths from au{o deaths vs from train deaths 300 
50000 100 

• Ignoring sample size effects - 4/5 vs 20/25 

• Reasoning by representativeness 

Problems arise when we judge how likely something is by estimating how representative it is of some 
class. 



When you're young, it comes naturally, but when you get a little older, you have to rely on mnemonics. " 

Suppose my schizophrenic patient comes in and seems to be deteriorating. I increase the neuroleptic a 
bit. At the next visit the patient is better. I am likely to see that as representative of cases in which a 
neuroleptic reduced symptoms. My judgement is likely to be based too much on that similarity and not 
enough on the probability of variation in the patient's clinical state from one visit to the next. You might 
not think I would make such a naive error: however, for many years psychiatrists generally used doses of 
neuroleptics well in excess of what was in fact needed to produce maximum benefit. 
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The Robust Beauty of Imprest Linear Models This heading is the titS^f a 
paper by Robyn Dawes. In it Dawes reviews what is known about the 
predictions of experts compared to mathematical predictions. 

The linear model referred to means a simple mathematical formula used to 
calculate an answer. An example might be a formula for deciding if a 
patient should receive depot neuroleptics. 

Depot score = resistance rating X 4 + disorganization rating X 1 + family 
support rating X 2 

A proper linear model has scientifically determined weighting factors for 
each variable (4,1 and 2 in the example). An improper linear model has no 
weightings or weighting factors chosen without solid scientific data. 

Through studies involving experts in many fields it is clear that experts can identify the important factors 
for making decisions, however, they virtually always are not as good at making decisions as are simple 
mathematical models based on the factors the experts identify! 

This points out that experts are good at knowing what is important to make decisions but like all human 
beings, they are poor at processing and combining probabilistic information. 

A good example comes from a study of pathologists who rated biopsies of patients with Hodgkins 
disease. They rated several factors about the slides and made an overall rating of severity for each 
patient. Their severity ratings did not predict survival. However, a regression analysis using their ratings 
of the various factors on the slides, was able to predict survival The pathologists knew what was 
important and could rate the factors but they could not extract maximum usefulness from it. A 
mathematical model did that better. 

One might say that a regression analysis is very sophisticated and not usable in many real world decision 
making situations. Maybe you just have impressions about the weight which various facts should be 
given in a decision - or maybe you can't even guess at the proper weight but just think a fact is a 
predictor of outcome. This is not a situation for scientific despair. Many studies show that even 
mathematical models based on no weighting factors (You just add up the facts by counting them.) or 
weighting factors based on clinical hunches produce better outcomes than experts can. These are the 
beautiful, though somewhat professionally threatening "improper linear models 1 ' referred to above. 

Sub-Optimal Treatment Algorithms 

Treating patients is not a very good setting for refining treatment algorithms. In clinical situations it is 
usual for sub-optimal treatment algorithms to survive - even thrive (Einhorn, 1982). Our goal of helping 
patients get better often conflicts with a goal of understanding what makes our treatments succeed or 
fail. Decision makers generally have much more confidence in their strategies than is scientifically 
warranted. One reason for that is that positive results are noted much more strongly than negative 
outcomes - making it more difficult to discern the true pattern of cause and effect. 

Clinicians cannot collect enough instances of each decision point in their treatment algorithms to assess 
the probability of success at each point. 
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Examples of sub-optimal al^fehms are the long prevalent use of excJ^ve doses of neuroleptics and 
the less than optimal doses on antidepressants which were recommended for preventing recurrence for k 
depression for about two decades. Sub-optimal treatments like these occurred in part because experts 
lacked the knowledge of what was the best algorithm, and they persisted because clinicians could 
perceive that patients did better with these treatments than with no treatment - but they could not 
perceive that there were still better ways to the use the treatment tools at hand. 

Advantages of Algorithms 



An algorithm is carefully reasoned out by experts who have full command of research findings and 
extensive clinical experience. The experts can take the time to weed out places where faulty strategy 
might occur. 

An algorithm attempts to most carefully translate research results into advice about specific situations. 
Linden (1994) has pointed out how treacherous it is to get from experimental outcome data to clinical 
recommendations. A research treatment protocol reveals outcomes for only a highly selective patient 
group; generalizability is a matter of speculation. The research usually addresses only one treatment 
intervention without any feedback of interim results - unlike clinical situations. 

Outcome studies should regard: 

Multi step process of real clinical treatment 
Time 

Feedback during treatment process 
Preferences of patient and therapist 

Linden says "it would be a major step forward to take into account the following: (a) instead of seeing 
treatment as a one-step event, time should be included as an important variable, and treatment be 
conceptualized as a process; (b) instead of only one treatment intervention, different options should be 
allowed in the treatment process; © not only the psychopathology, but also the motivational" 
variables, evinced by therapist and patient, should be recognized as factors that influence the treatment 
process." 

He believes that clinical research will be greatly aided by research which tests decision rules or 
"intentions" in an ongoing treatment process rather than just the usual highly controlled single 
intervention approach. 

Linden's very high standards for what treatment algorithms should do is an appropriate transition to a 
discussion of computerized algorithms. 

What Should a Computerized Algorithm Do? 

A computerized algorithm should incorporate much greater complexity and subtlety of problem analysis 
and recommendations (without overwhelming the user) than a paper based algorithm. 

It should be easy to update. Soon the standard will be continuous automatic updating via modem 
connections to the algorithm's authors. 

It should handle a wide variety of cases. Help is needed most for unusual cases - not garden variety 
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cases, so the program must olRbust. 

It should address patients' preferences, fears and tolerance or intolerance of various side effects. It should 
take into account comorbid conditions and history of prior treatment. It should take account of the 
psychiatrists's preferences and experience when appropriate. 

The psychiatrist and patient's preferences regarding urgency of symptom relief, risk of side effects, and 
therapeutic failure - such as temporary worsening of symptoms - on the way to gaining long term better 
outcome should be considered. 

At the edges of its sphere of expertise, it should be able to "fail gracefully" - i.e. be able to give a 
response that is at least not bad advice. At best it should recognize when it is dealing with a case which it 
was not prepared to handle. 

It should teach approaches that can be remembered and applied to other cases. 
How to Computerize an Algorithm 

Programmer and Expert Collaboration The programmer and expert need to work closely together for 
weeks or months to codify the expert's knowledge in well defined rules. In that process the expert often 
has to think out loud about how he solves particular problems and then work with the programmer to 
form those insights into comprehensive rules. Many attempts at this have failed because the expert was 
not sufficiently committed to the project and therefore would not devote sufficient time, or because the 
expert cannot articulate his strategies. 

Considerable time has to be spent making sure all possible situations are covered by the algorithm. No 
condition can be left unspecified. 

Computerize We are focusing today on treatment algorithms based on rules, but other approaches are 
possible. Computerized neural networks learn from data and outcomes you supply. The network creates 
its own "rules" for making decisions. There are also case based reasoning and genetic algorithm 
approaches. Fuzzy logic is a way to help deal with uncertainty in algorithms. 

A major challenge in computerizing the algorithm is to make it easy for the user to get the information 
without spending much time learning the program or searching. 

Test the Program Some people would insist that a program should be rigorously experimentally tested 
before release. In the real world that is not practical because of the costs involved. I think it is 
appropriate to expect a computerized algorithm to meet standards expected of other educational 
materials. An algorithm may be thought of as a kind of meta-analysis performed by the expert authors. It 
would be desirable for computerized algorithms to undergo peer review similar to that for traditional 
journal publications. 

Distribute Distribution should be straight forward, however some interesting sociological issues will 
arise. Since algorithms provide more concise statements of how to treat a condition than do textbooks 
and ordinary journal articles, non-psychiatrist physicians, psychologists, social workers and nurses will 
want to use them. I think we will find people increasingly well informed and asking us questions about 
- treatment strategy based on their examination of the treatment algorithms. 
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Patients and their families wlKegin to consult the algorithms. The InHRet will greatly facilitate this 
because it permits an interested person to explore a treatment algorithm published on the Net anywhere 
in the world. It will of course be up-to-date - unlike a book or magazine article. A year ago I saw my first 
patient who had checked out fluoxetine on the Internet in anticipation of getting it for his depression and 
last week for the first time a patient requested a prescription refill over the Internet! 

Computerized algorithms will tend to demystify what we do as psychopharmacologists. That will require 
some changes in the way we perform as healers. 

What Will Convince Clinicians to Use Computerized Algorithms? 

The approach should be that if you build a better mouse trap people will want to use it. 
Here is a wish list of characteristics for a user friendly computerized algorithm: 

Make it readily apparent that the program can deliver on the promise of aiding the clinician in caring for 
patients. It must hot be just good enough to help a green resident and must be fully up- to-date on the 
latest developments - those that an average competent psychiatrist may not have had time to learn about. 

Give it a high level of expertise including how to handle difficult cases. 

Give it a pleasing interface - which is familiar and easy to use. Presently, Macintosh, Windows and Web 
page interfaces do this best. In the near future a modem connection to its authors will probably be 
considered essential. 

Associate it with other useful programs such as a hospital information system, prescribing information or 
a record keeping system so that it is very convenient to access. 

Make it work very quickly. Tedious data entry will not be tolerated by most clinicians. 

Make sure it avoids the "Greek oracle" approach - in which an inscrutable master renders decisions 
without explanation (Shortliffe, 1994). 

Do not make users feel they are traversing a maze. The user should be able to see where he is in the 
consultation as it progresses. It should allow many ways to see the expertise captured in the program. 
The user should be able to see the impact on the outcome when a fact is changed. Obviously, the 
research and clinical experience basis for recommendations needs to be spelled out. 

Will Algorithms Stifle Clinicians? 

More complete and more easily usable knowledge about treatment steps should be liberating. It will 
reduce by a little the uncertainty we deal with. It can free us to devote more attention to other elements 
of good treatment such as educating the patient, building rapport and gaining treatment compliance. 

Even when you chose not to follow an algorithm's recommendation you will be able to do it with good 
knowledge of how daring your contemplated variation is and with the algorithm at hand you can 
probably more clearly document why you have chosen an alternative approach. If you are electronically 
transmitting your experience to a feed back system for that algorithm - as we will discuss next - your 
variation will uniquely contribute to the further development of the algorithm. 



7 of 10 



11/2/99 2:55 PN 



http :/Avww.m hc.com/algorith. htm 




Refining Algorithms Through Networked Users Groups 

Thousands of psychiatrists try treatments and get results every day but those mini-experiments contribute 
little to general knowledge because there is no way to bring the results together in order to see patterns. 

Computerized algorithms offer the possibility of gathering outcome data on thousands of therapeutic 
trials without extra work from the psychiatrist. Feedback from nation-wide experience could be a major 
reward for psychiatrists who use computerized algorithms. 

Imagine being able to use your computer connected through the Internet to a center that supports 
algorithms for treating schizophrenia. Using the program, you describe your patient and the program 
gives you a recommendation about what to do. It can be as up-to-date as of today because experts are 
maintaining the system. The program might tell you that the program's experience to date is 500 
recommendations at this specific spot in the algorithm. Two hundred patients had very good outcomes 
from that recommendation; 200 more tried it but either had to stop the treatment or had a poor outcome - 
you could see the breakdown of this if you want; and 50 psychiatrists declined to follow the 
recommendation and you can see what they said about their thinking and what their outcomes were; 50 
patients declined to try the treatment or were lost to follow up - a breakdown of that group would also be 
available. 

With information like this pouring in, I can imagine experts putting up things equivalent to road signs 
like: "This recommendation is very solid - don't think about not following it"; or "User experience in 
following this recommendation is making it look doubtful. We are reassessing it". 

There should not be just one system like this for schizophrenia treatment algorithms - there should be 
several competing ones. With hard data coming in through user feedback, the best algorithm will prove 
itself in time. But of course, like other complex systems, one group's algorithm might pull ahead in 
treating first break schizophrenics and another might excel at treating people with comorbid retardation. 
This could lead to hybridization, mergers, buyouts etc. among the systems - to the benefit of patients and 
clinicians! 

Such a system seems likely to yield substantial improvement in treatment outcomes. This can be 
anticipated from the research which shows that, throughout medicine, improved treatment methods are 
slow to be adopted. An example in field of schizophrenia treatment is clozapine. Today clozapine is 
being given to only a modest proportion of the patients whom experts would recommend it for. The 
adoption of the practice recommended by experts has been slow. The psychological and economic costs 
of this slow adoption of new treatments is very large. 

Even in situations where algorithms achieve outcome improvement of only a few percentage points, the 
human and economic benefits can be large. Those smaller improvements may well be at a level which a 
clinician treating an average patient load may not be able to detect. Suppose that by using an improved * 
algorithm you could achieve a 2 - 3 or maybe 5 percent better treatment ouLwme, You would probubly 
not be able to detect such a small change by just reviewing your patients, It would be detectable among a 
network of psychiatrists using the treatment algorithm and recording their results. 

Suppose following a particular algorithm could achieve a $100/year saving for each schizophrenic 
patient. Maybe the saving would be achieved through less hospitalizations, or greater earnings at jobs 
your patients gets. If enough psychiatrists used the algorithm so 1 in ten patients in treatment got the 
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benefit. The cost saving nati^lly would be about $12.5 million! Tha^^ssuming only half of all 
schizophrenic patients are in treatment and one tenth of those receive the benefit of our algorithm. If all 
schizophrenics got the benefit the saving would be $125 million. 




"How do you want it — the crystal mumbo-jumbo or statistical probability?" 
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